This issue of Chemistry in Australia features a report describing how a lab used AI to design and carry out a chemistry experiment in just minutes. It’s a fascinating example of how AI can accelerate “the pace and number of scientific discoveries, as well as improve the replicability and reliability of experimental results”. This is one of the “good” AI stories.

Balanced against this are the “bad” AI narratives, and some of these are already being witnessed. For instance, last November a team from the Lawrence Berkeley National Laboratory in the US reported the discovery of 41 novel inorganic compounds in just 17 days. However, by early January an analysis published as a preprint on ChemRxiv pointed out “four common shortfalls in the analysis. These errors unfortunately lead to the conclusion that no new materials have been discovered in that work ... The predicted compounds investigated herein have all their elemental components located on distinct crystallographic positions, but in reality, elements can share crystallographic sites”. As a result, “two-thirds of the claimed successful materials ... are likely to be known, compositionally disordered versions of the predicted, ordered compounds”.

Co-author Professor Gerbrand Ceder defended the study on LinkedIn: “We have no doubt that a human can perform a higher-quality refinement on these samples”, he wrote. “However, it was our objective to show what an autonomous laboratory can achieve.”

The problem with this statement is the authority that a peer-reviewed paper in Nature provides this “experiment within an experiment”. Ceder’s group may have shown what AI can achieve, and also what it can’t, but when we see chemistry claims published in Nature we expect something more reputable than the output of ChatGPT. Human intelligence must not be forfeited.

Elsewhere in the literature, AI is undermining the authority of scientific research through the proliferation of “paper mills”, which generate fake research papers for people who want to boost the publication count on their CVs. According to Nature: “There are hundreds of thousands of bogus ‘paper-mill’ articles lurking in the literature”.

Ironically, AI programs are being employed to detect fake papers, but they will always be playing catch-up. Meanwhile, the trustworthiness of science and the integrity and replicability of the research record diminishes.